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ABSTRACT 


Lecture material of a sophomore large-enrollment (N=271) system 
programming 15-week class was delivered solely online using a 
new video-based web platform. The platform provided accurate ac- 
cessible transcriptions and captioning plus a custom text-searchable 
interface to rapidly find relevant video moments from the entire 
course. The system logged student searching and viewing behav- 
iors as fine-grained web browser interaction events including full- 
screen-switching, loss-of-focus, incremental searching events, and 
continued-video-watching events with the latter at 15-second gran- 
ularity. Student learning behaviors and findings from three research 
questions are presented using individual-level performance and 
interaction data. Firstly, we report on learning outcomes from al- 
ternative learning paths that arise from the course’s application 
of Universal Design for Learning principles. Secondly, final exam 
performance was equal or better to prior semesters that utilized 
traditional in-person live lectures. Thirdly, learning outcomes of 
low and high performing students were analyzed independently by 
grouping students into four quartiles based on their non-final-exam 
course performance of programming assignments and quizzes. We 
introduce and justify an empirically-defined qualification threshold 
for sufficient video minutes viewed for each group. In all quartiles, 
students who watched an above-threshold of video minutes im- 
proved their in-group final exam performance (ranging from +6% 
to +14%) with the largest gain for the lowest-performing quartile. 
The improvement was similar in magnitude for all groups when 
expressed as a fraction of unrewarded final exam points. Overall, 
the study presents and evaluates how learner use of online video 
using ClassTranscribe predicts course performance and positive 
learning outcomes. 
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1 INTRODUCTION 


The primary goal of this paper is to report on and evaluate the re- 
placement of physical lectures (that used a blend of active learning 
and instructor-led live-coding examples) with online content de- 
signed for online viewing. These videos were delivered using a new 
custom web-based, text-searchable video player system, ClassTran- 
scribe, that met accessibility standards, applied Universal Design 
for Learning principles, and allowed students to search for relevant 
content by indexing transcribed video. The ClassTranscribe applica- 
tion generated low-cost accurate captions by combining automated 
and crowd-sourced techniques. The latter included editing by both 
course staff and students within the course. Learning outcomes 
of the Spring 2019 students were compared to those of previous 
comparable semesters. By combining behavioral interaction data 
captured by ClassTranscribe with student “gradebook” assessment 
data, the learning benefits of the tool and online videos were exam- 
ined by class quartile within the course and also compared to three 
previous comparable semesters. 

In Spring 2019, Monday-Wednesday-Friday lectures for a sopho- 
more system programming class for CS majors were scheduled at 
8am. The instructor suggested this early time would severely im- 
pact attendance and learning. Rather than penalizing absence, the 
instructor offered to record equivalent lecture content specifically 
designed for online viewing. The scheduled time slots were used 
three times: once for the first lecture, once for a guest lecture, and 
once for the closing lecture. The weekly Wednesday lecture time 
was re-purposed for instructor office hours, which were utilized 
by two students over the semester, and the the Monday and Friday 
meeting times were not used. This change was announced only on 
the first day of class. The instructor offered to provide in-person 


8 am lectures for students who requested an in-class experience, 
but no student requested it. Previous semesters also offered on- 
line video lectures, but the content consisted of simple automated 
recordings of the classroom lectures. For this semester, only the 
new recorded video lectures were used by the course and delivered 
by the tool presented in this paper. 

This paper is organized as follows. In the background section, we 
review the use of lecture videos and Universal Design for Learning 
(UDL) principles from both the CS education and educational liter- 
ature, and we briefly discuss students’ use of videos and searching 
for content as a means of regulating their metacognitive awareness. 
In the related work section, we introduce the pedagogical utility 
of the new tool by enumerating its unique features that are only 
sparsely represented in other video delivery systems. In the technol- 
ogy aspects section, we describe interesting technical details from 
the tool development activities and platform IT requirements. In the 
learning outcomes section, we evaluate the effect on student learn- 
ing outcomes and utilization of UDL resources. Limitations and 
opportunities for further work are discussed in context throughout 
the paper. 


2 BACKGROUND 


Online learning environments constitute an important application 
area of human-computer interaction and are ubiquitous in today’s 
educational landscape. A key instructional element in these learn- 
ing environments is the set of video lectures that often serve as 
the unique portal connecting the instructor to the learner, apart 
from serving as primary portals of content delivery [18, 20, 23]. Not 
surprisingly, studies have found that it is the course component 
with which students spend most of their time [3, 29]. Other studies, 
especially those on massively open online courses (MOOCs), have 
found that a significant number of students primarily watch only 
videos, while skipping over assessment problems and other inter- 
active course components [1, 15]. Students report that the videos 
make them feel more engaged in the course and motivate them[10], 
and that they also feel more connected to the instructor [13, 26]. 
Additionally, students perceive that the videos help them learn [27]. 
However evaluating if the videos have actual effects on learning is 
a necessary criteria for most practitioners, and this paper investi- 
gates video watching beyond affective measures and perceptions 
to better understand learning outcomes, specifically in videos that 
are closed-captioned and transcribed. 

Videos used in the study were closed-captioned and had ac- 
companying transcriptions, which adhere to Universal Design for 
Learning (UDL) principles. Following these principles ensures that 
students’ diverse abilities and needs are met. A UDL framework, 
[5] suggests that instructional materials encourage (1) equitable 
use, (2) flexibility in use, (3) simple and intuitive use, (4) perceptible 
information, (5) tolerance for error, (6) low physical effort, and (7) 
adequate size and space for use. By providing materials that in- 
corporate these principles, students of all backgrounds have fewer 
barriers to learning [25]. Although research has found that students 
enjoy courses that adhere to UDL principles (e.g., [22]) and that 
they perceive their instructor’s teaching abilities to be better[8], 
there do not seem to be significant differences in learning outcomes 
for students with and without disabilities. For example, [14] found 


no significant difference in both disabled and non-disabled students’ 
learning between a course taught using UDL principles compared 
to a course taught in a regular manner. [9] found improvements in 
learning outcomes for some types of UDL materials but not all. This 
study includes the UDL materials, in the form of videos accompa- 
nied with text, and secondary learning opportunities of the online 
course book, to understand if this material promotes better learning 
outcomes for all, and if alternative learners paths that utilize these 
materials result in positive learner outcomes. 

One reason that we hypothesize that videos adhering to UDL 
principles — and videos that have a text-searchable feature in par- 
ticular — will be beneficial for all students is because of their ability 
to assist in students’ metacognitive awareness. Metacognition is 
awareness of one’s own thinking; more specifically, it enables a 
person to monitor, assess, and regulate one’s understanding and 
thought process [11]. Thus students who are metacognitively aware 
are able to recognize what they do not know and then figure out 
how to extend their knowledge [2]. As such, engaging in metacogni- 
tive thinking and strategies improves learning outcomes [19, 21, 34], 
and this effect on learning outcomes also holds true in online en- 
vironments [4]. Engaging in metacognitive strategies is effective 
for improving low-achievers’ learning outcomes [6], and it also 
explains the difference between high and low achievers[17, 33]. 
Instructors must be deliberate in their support of metacognitive 
awareness [28]. In this particular course, students had access to 
a unique text-searchable interface to rapidly find relevant video 
moments from the entire course. Thus, they were able to reflect on 
what they did not understand and could easily locate material to 
assist them in clarifying that understanding. [32] developed a simi- 
lar tool, although no research to date has looked at the relationship 
between the tool’s use and student performance—a gap in which 
this study attempts to fill. In the next section we outline related 
tools and discuss the utility of the tool presented here. 


2.1 Related Work 


Modern creation and delivery of educational online lecture videos 
includes a multitude of presentation formats, sources, and delivery 
tools. Sources include lecture-room multi-view recording systems 
to simultaneously capture live lectures from room camera and 
display sources, screen recording applications (e.g. Camtasia, OBS 
Studio and native support in Microsoft Windows 10), and content 
professionally edited for MOOCS and large audiences. Delivery 
mechanisms span emailing a hyperlink to the class (“Here’s the mp4 
file on my shared directory”) to sophisticated commercial systems 
that optimize video playback for large audiences (e.g. YouTube, 
Echo360, Kaltura, Coursera). 

Recent work on crowdsourcing transcriptions include [16, 24, 32]. 
The system presented here overcomes limitations from existing 
open-source and commercial alternatives. Namely, it is a system 
that enables all of the following features: 


e Has inexpensive and accurate captions. Commercial caption 
systems offered free-but-low-accuracy transcriptions, or pro- 
fessional transcriptions services at $1 per video-minute. 

e Has fine-grained server-side logging of user interaction events 
to support educational research, student engagement and 


course rewards based on students exhibiting desired behav- 
iors. The former includes both insights into how and when 
students are engaging with course videos resources and, by 
joining with course performance data, provides insights into 
how different learning behaviors benefit each student or 
student sub-populations. 

e Deploys easily on University hardware or the commercial 
cloud. 

e Is open source and extendable. The source-code is available 
at https://github.com/cs-education/classTranscribe. 

e Supports modern accessibility web standards (e.g. closed- 
captions with configurable display settings, Aria-tag support 
for blind users) 

e Supports multi-stream and viewer-controlled playback speed. 

Enables the finding of relevant content by indexing tran- 

scriptions within the current video and across the entire 

course that is relevant to the student’s current activity or 
the student’s pre-exam review knowledge-seeking question. 


In the tool presented here, audio is initially transcribed using 
a modern high-accuracy speech-to-text cloud service (Azure Cog- 
nitive Services Speech-To-Text) at a cost of approximately $1 per 
hour. This service also supports improved transcription accuracy 
of domain-specific speech (though this feature was not enabled 
or evaluated in this study, and is an opportunity for future tool 
improvements and research). 

A similar comparable system is the Indexed Captioned Search- 
able Videos (“ICS”) system used at the University of Houston [32] 
and [31]. Recent evaluation of ICS in an introductory psychology 
course [30] reported increased instructor and student satisfaction 
but “confounding variables in the structuring of the course prevent 
us from making firm conclusions about student performance.” 


3 TECHNOLOGY ASPECTS 


An early version of the software that included automated speech 
to text was first available in November 2018. Course videos were 
automatically enumerated and downloaded by polling playlists 
from an existing commercial lecture capture system. However this 
paper reports on a Spring 2019 system programming course for 
CS majors where ClassTranscribe was used exclusively to deliver 
lecture content designed for online viewing. The lecture videos 
closely mirrored the lecture exposition and problem-solving lecture 
material used in previous semesters. 

The Web accessibility of ClassTranscribe was evaluated using 
the AInspector WCAG Firefox plugin [12], and a University web- 
accessibility reviewer also manually reviewed it. Specific issues 
found were entered as bug reports. Correcting accessibility errors 
were useful and appropriate bug-fixing activities for new program- 
mers to a project because they had the following characteristics: 
i) Shallow, front-end errors that were usually specific to a specific 
page or single resource; ii) Easy to find, reproduce, test and fix 
iii) Motivating and small iv) Had a high perceived value to new 
developers v) Encouraged developers to appreciate accessibility 
concerns from the earliest involvement with the project. 

The system programming lecture videos were recorded using 
the free, cross-platform OBS Studio software using the H.264 MPEG 
encoder at “High quality Medium Filesize” application settings. OBS 


Studio was configured with two recording layouts that could be 
switched while recording: Head-shot from the inbuilt laptop cam- 
era, or laptop full-screen with a small “postage stamp” head-shot 
inlay. The former was only used for introductory and closing re- 
marks. A traditional 50-minute lecture was recorded as a series of 
small problem-centered video segments (typically 4 to 10 minutes). 
No post-editing was performed, as the instructor believed it was 
quicker to re-record the segment than fix errors. The total semes- 
ter content for the course compromised of 37.0 video hours and 
5.2 GB of 286 mp4 files. Bandwidth requirements of introductory 
full-screen talking head format were 200 KB/s, however the bulk 
of the content — screen annotations and sketches (created using 
ScreenBrush.app for OSX and a Wacom tablet pen-based input de- 
vice) and developing program code using a text editor and terminal 
shell — compressed to an average 40 KB/s. 

The technology of the ClassTranscribe system was comprised 
of common HTML 5 components (Bootstrap and jQuery), Node.js 
web server, and a SQL database. The system was built and deployed 
as a set of Docker containers on a dedicated Linux virtual machine. 
The database schema and design choices were published in [16]. 

The video server was deployed to a single virtual machine hosted 
on-campus on a DL380 Gen 8 2014 server with 1 Gbit/s network 
connectivity, 12 CPU cores, 15 GB RAM, 500 GB SSD storage, with 
the expectation that this would be sufficient to support at least 
500 concurrent users. In practice, due to the course’s small video 
file size and bandwidth (typically 10 - 20 MB and 40 KB/s respec- 
tively), neither network, storage, or CPU needs of a single host 
machine were limiting factors. Content for other courses (including 
a computer science theory course, statistics, and machine learning 
course) from lecture-capture sources had larger files and bandwidth 
requirements,> 200 MB, 4 MB/s respectively, but comprised only 
5% of the total video minutes viewed. A future version of the sys- 
tem could also pre-process external large files to be smaller, which 
would also benefit mobile users on low bandwidth connections. 


4 LEARNING OUTCOMES 
4.1 Data Collected 


Table 1: A summary of the 1198413 events by user action. 


Action Count Fraction 
timeupdate 413912 0.345383 
play 164428 0.137205 
seeking 160924 0.134281 
pause 153034 0.127697 
userinactive 134455 =0.112194 
seeked 102531 0.085556 
changevideo 24750 0.020652 
changedspeed 23151 0.019318 
fullscreenchange 17195 0.014348 
selectcourse 2850 0.002378 
filtertrans 1087 0.000907 
edittrans 83 0.000069 
sharelink 13. 0.000011 


80000 


60000 

40000 

- | | 
Lavaaal 


23 4 5 6 7 8 Q9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 
Local Time (hour) 


Number of Events 


Figure 1: Events captured by hour 


Event logging of student interactions was available in approxi- 
mately the last 7 weeks prior to the final exam, of which 6 weeks 
contained lecture video content. Table 1 presents a summary of the 
events collected for the system programming course by event type. 
A timeupdate event was generated whenever a user watched 15 
continuous seconds of video, which could take fewer wall-time sec- 
onds if the video was played at a higher speed. A total of 1725 video 
hours was served over the last 7 weeks of the course. The small 
number of selectcourse events suggest students remained logged in 
between viewings. The small number of times that students shared 
a URL link to a specific moment in the video suggests students 
may have been unaware that they could easily share and discuss a 
particular video moment in the course’s discussion forum external 
to this tool. This suggests a future version of the tool should present 
user tips to enable more pedagogically valuable utilization of the 
tool. Students used the tool throughout the day, particularly in the 
afternoon and evening (See Figure-1). Only 0.6 % students chose to 
learn at the scheduled 8am time slot. 


4.2 Evaluation 


In the second part of the paper, we use behavioral and gradebook 
data to understand how students learn and thrive in the course. We 
present three student-outcome questions - 


e “Are students exploiting multiple learning paths?” Are stu- 
dents utilizing the same learning resources in similar propor- 
tions, or are they exploiting multiple learning paths available 
and thriving? 

e “First, do no harm” - Did the cancellation of traditional in- 
person lecture and sole use of the ClassTranscribe for lecture 
content, affect exam performance when compared to similar 
previous semesters? 

e “Who benefits?” Do online lecture videos help all students, or 
only the strongest students? Are more video minutes viewed 
associated with improved exam scores? 


Table 2: Final exam score (out of 100) for students who chose 
to learn without using ClassTranscribe (Path-I), watched be- 
low average duration of video content in ClassTranscribe 
(Path-II), or above average duration (Path-III). 


Learner Path-I 
n = 78(28.8%) 


Learner Path-II Learner Path-III 
n= 99(36.5%) n= 94(34.7%) 


0 minutes 1 to 499 minutes > 500 minutes 
Qy 57.0 59.8 76.5 
Q2 70.4 71.4 82.4 
Q3 77.8 79.4 88.8 


4.3 Learning Outcomes of Multiple Learning 
Paths 


Students may learn about a particular system programming topic 
by attending a lecture (2015-2018), viewing a recorded lecture from 
a commercial capture system (2015-2018), viewing lecture video 
recorded for online use with ClassTranscribe (in Spring 2019 se- 
mester only), searching and/or reading lecture transcript (Spring 
2019 semester only), reading the free course book in a variety of 
formats (epub,pdf,html; in all semesters), or viewing recordings of 
classroom lectures (previous semesters only). This is an example of 
application of UDL principles, where multiple learning activities 
and resources can be utilized by the student to achieve the same 
learning goal. Using the behavioral data captured in Spring 2019, 
it became possible to compare learning outcomes of alternative 
learning activities and validate student use of UDL materials in the 
course’s design. 

In Spring 2019 semester, out of 271 students taking the final exam, 
194 (71.6%) students used ClassTranscribe and 77 students never 
viewed videos in ClassTranscribe (28.4%). This group was identified 
as the “Course Book” learners, as this was the most likely source 
that the students used to learn the content. It is possible a subset 
of students found and viewed previous semester lecture videos on 
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Figure 2: Histogram of final exam scores for different learn- 
ing paths. 


the commercial lecture capture system, or co-watched the intended 
videos while another had logged in. The online course book is the 
most plausible learning path because it was positively mentioned 
in course feedback forms and linked from the course web pages. 
The 194 students who viewed at least 15 seconds of continuous 
video in ClassTranscribe (i.e. generated at least one timeupdate 
event) watched an average of 500.1 video minutes (equivalent to 8.3 
video hours, and was rounded to 500 video minutes in subsequent 
analyses). Behavioral data were captured over 7 weeks, of which 6 
contained content, corresponding to 83 minutes viewed per week. 
This time-on-task is lower than 3 live lectures total 150 minutes 
per week. 

Students’ primary choice of learning activity were characterized 
as 3 learning paths: i) No use of video lectures (i.e., the course book 
readers) ii) Below average use of video lectures iii) Above average 
use of video lectures. Reading events of the course book were not 
available. A limitation of this analysis is students may have further 
supplemented these expected learning activities by engaging in 
other relevant but untracked resources (for example, web searches, 
discussion forums, and discussions with their peers and course 
staff). 

The final exam quartiles and histogram of the learning outcomes 
of the 3 learner paths are summarized in Table-2 and Figure-2 respec- 
tively. The primary finding was above average usage of ClassTran- 
scribe led to improved exam performance. For example the median 
of Path-III learners was > 11.0 point improvement on the final 
exam compared to learners choosing Path-I or Path-II. 

A secondary finding was that a significant fraction of students 
can still earn competent — albeit on average lower — scores in a final 
exam by choosing alternative and secondary learning resources 
(Learner Path-I). This is encouraging feedback and validation for 
Universal Design advocates and practitioners who have created, or 
are considering creating, equivalent course resources in alternative 
formats. 


4.4 Comparison With Prior Semesters 


The final exam over all previous semesters used 45 randomized 
multiple choice questions (with randomized variants) that covered 
all topic areas of the course, and included concept questions and 
skilled application of a system programming idea within a program- 
ming context. Long form questions varied by semester, were graded 
manually using a rubric, and were excluded in this analysis. The 
exam scores of the Spring 2019 semester were compared to previous 
Spring semesters with the same instructor. Though programming 
and lab assignments have varied incrementally over time, the over- 
all content, structure, lecture delivery and exam assessments have 
remained similar. The quartiles of multiple choice final exam scores 
are presented in Table-3. All quartiles of the Spring 2019 semester 
showed a modest improvement compared to previous semesters. A 
counter argument is to observe that quartiles show a mostly slow 
rising trend, and we hypothesize that active and blended learning 
in physical lectures or use of lectures videos has an insignificant 
role in this course. In either perspective, we can still conclude that 
the dropping of lectures caused no significant harm to exam perfor- 
mance ina lower-curriculum required class of the CS undergraduate 
program. 


Can we therefore continue and conclude that lectures have no 
other intrinsic or extrinsic additional value? No. A limitation of 
this finding is that we have reduced learning outcomes and lecture 
value to the numerical performance in a final exam setting and not 
measured a student’s experience and affect for Computer Science 
e.g. long-term interest, motivation, perseverance, and the ability to 
thrive in future academic and non-academic settings. 


Table 3: Final exam multiple choice quartiles and 50% spread 
(Q3 — Q1), for students in Spring semesters of 2015,2016,2018, 
and 2019. 


Statistic 2015 2016 2018 2019 


N=195 N=325 N=247 N= 271 
Q1 60.1 58.7 60.0 63.6 
Q2 (median) 68.1 69.6 71.1 72.7 
Q3 76.6 78.3 80.0 83.3 
O50; 16.5 19.6 20.0 19.7 


4.5 Learning Outcomes By Quartile 


Lower performing students (Q; Q2 ) may struggle to succeed due to 
diverse course-related, affect, and acquisition reasons (e.g. missing 
prerequisite knowledge or skills, incomplete, ineffective or ineffi- 
cient learning strategies), as well as external non-course hardships 
(e.g. financial stress, limited time, reduced peer support, lower con- 
fidence, imposter syndrome). Apriori, i) It was unclear if ClassTran- 
scribe could be used effectively by lower-performing students (com- 
pared to their peers) to improve their course-exit performance as 
measured by their final exam score; ii) However, the tool provided 
an opportunity for students to find and review relevant lecture 
material on any day and any hour of the day. Would this help 
ameliorate difficulties experienced by lower performing students? 
Similarly, could upper-performing students also gain from its use, 
compared to their peers? 

The last analysis presented here explored if learning benefits 
were possible for both lower performing and upper performing stu- 
dents in the Spring 2019 semester. Students were grouped into per- 
formance quartiles based on their total weighted course score (with 
the final exam score excluded). The course score was comprised of 
numerous assessments that is typical of a CS programming-heavy 
course: challenging autograded weekly programming assignments, 
quizzes and programming problems given under exam conditions, 
and lab programming exercises. All assessments were individual 
work. 

The total class enrollment (N=271 exam takers) was sufficiently 
large to evaluate 4 quartile groups each split into 2 subgroup con- 
ditions (below vs. above threshold of total-video-minutes-viewed). 
To compare ClassTranscribe use within each group, the average 
of video-minutes-watched-by-that-group-only was an appropri- 
ate threshold because it i) is a simple general definition that can 
be implemented in future analyses and replication studies; ii) is 
statistically expedient and likely to ensure a reasonable subgroup 
population size; iii) allows the comparison of student outcomes 
for students of a similar competency. Using a threshold based on 
total class average or fraction based on total video minutes were 


Table 4: Within-quartile final exam improvement for each course performance quartile. For each quartile the exam mean and 
standard deviation are reported for below and above average minutes-viewed. 


Course Performance Threshold Exam Exam Mann Whitney Peer Improvement Fractional Reduction 
Ai 

Q; (Rank) ti (hours) (score;(t < t;)) score;(t = T)) Pp Absolute A; a0 (scara) 

Q, (Rank 0% — 25%) 5.6 55.4 + 16.3 69.9 + 16.9 0.005 14.4 0.38 

Q2 (Rank 25% — 50%) 6.2 68.6 + 12.2 78.6 + 8.6 0.002 10.1 0.39 

Q3 (Rank 50% — 75%) 9.2 74.1 + 10.4 80.4 + 7.7 0.008 6.3 0.30 

Q4 (Rank 75% — 100%) 11.3 81.1+ 10.1 86.2 + 8.5 0.029 5.2 0.38 
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Figure 3: Overall Course Performance partitioned by Quar- 
tile Rank 


considered but rejected in favor of a per-quartile threshold due to 
the above 3 reasons. 

For each quartile, students who used ClassTranscribe more than 
average for that quartile, surpassed their peers’ exam performance 
in the same quartile by a significant amount (Table-4 and Figure 3)). 
For example, on average, in the lowest performing quartile, a stu- 
dent could increase their exam score by 14.4 absolute points (out 
of 100) compared to another student in the same quartile who did 
not significantly use the tool. A Mann-Whitney U test [7], which 
does not require the normal distribution assumption, rejected the 
null-hypothesis. The exam performance increase for every quartile 
is significant at the p < 0.05 level. 

The largest increase in exam improvement was exhibited by the 
lowest performing students (Qj) using least additional studying 
(5.6 hours over the 7 week period). This supports the finding that 
use of ClassTranscribe is an effective learning technique that is 
open to students of all competency levels. 

A second finding was that the benefit to all students was similar 
in range (30 — 40%) when expressed as fractional reduction of av- 
erage points lost on the final exam for each quartile. Replication 
of this fractional effect, understanding its magnitude, cause and 
scope, is worthy of future research. Further, a reasonable message 
to students of all abilities is, “Want to do well in the final? Take 


time to watch the videos over the semester and you can reduce 
your exam points you would have lost by a third” 


5 CONCLUDING REMARKS 


We removed lectures from a large enrollment system programming 
class and added online lecture content designed for online viewing, 
using an accessible video system with learning analytics. 

The fine-grained student behavioral data captured from the tool 
was used successfully to explore varied aspects of student learning. 
Learning outcomes both within the course and across compara- 
ble semesters provided evidence that use of video lectures using 
ClassTranscribe is an effective learning technique that leads to 
positive learning outcomes for students of all competency levels. 

Student behavioral data also provided insight into actual and 
successful student use of alternative learning paths available due 
to the application of Universal Design for Learning principles. 

This paper examined learning outcomes with respect to exam 
scores and found statistically significant effects with video minutes 
viewed. It is the first paper to report and evaluate learning outcomes 
of ClassTranscribe. The potential for future research and education 
insights using these (and similar future data) is broad and significant, 
including for example, machine learning models and multivariate 
statistical models to predict student outcomes and behaviors, and 
Hidden Markov Models to model event sequences. The results 
presented were based on a single measure - video minutes viewed. 
Future work will explore other learner behaviors that are possible 
with the data presented here (for example, engagement effects 
due to full-screen viewing, effects of search and transcription use, 
and comparing video minutes to wall-clock minutes to determine 
which is a better predictor of learning in technically dense computer 
science content). 

The results and experience of this study were also used to iden- 
tify limitations and prioritize future improvements to the tool (for 
example, encouraging greater use of search and share functionality 
and improving accessibility). 

Practitioners interested in replicating or adopting this approach 
may use the repository to deploy the system to their own server, or 
use the existing server at classtranscribe. illinois. edu which 
is supported by email, classtranscribe@illinois. edu. 
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